The present invention generally relates to computer-related technology, and more particularly to the use of biosignals of a user wishing to control a computer-controllable activity or operation, including computer games.
Brain-computer interface (BCI) or Neural Interface (NI) devices that fall into the general category of Biosignal Interface (BI) technology are gaining increasing importance for controlling electronic systems, a notable example of which is computers. Applications include biomedical appliances such as wheelchair and sailboat controls, as well as communication devices allowing, for example, conversion of eye positions to keystrokes of a word processing device. Other applications include biofeedback devices aimed at the control of emotional states, and NI devices to control computer games. In the broadest sense, even voice recognition can be considered as a biosignal interface.
Biopotentials generally result from the activity-dependent change of ionic composition of any cell's cytoplasm. In an idle state, all living cells are at a resting potential, typically −20 to −80 mV across their membranes versus the extracellular space. Excitation of any cell results in opening of selective ion channels, starting with fast sodium channels and calcium channels, allowing extracellular Na+ to enter the cell's cytoplasm and thereby depolarize the cell to a typical range of about +100 to about +150 mV compared to the extracellular fluid. If this type of excitation happens in multiple cells simultaneously, extracellular electrodes can sense the difference in charge and the resulting electrode output signals can be recorded. This type of biopotential and changes thereof are the basis for a variety of diagnostic tools, such as electrokardiogram (EKG), electromyogram (EMG) and electroencephalogram (EMG). The exploitation of biopotentials beyond the diagnostic applications is emerging in prosthetic limbs, where nerve signals can be measured and converted into control signals for governing mechanical movement of artificial limbs. In addition, biofeedback has been used for the purpose of facilitating meditation or preparing athletes for sporting events. A relatively new use of biosignals includes their use in computer games as a novel contribution to virtual reality sensation.
In the general field of using brain-based measurements as the source of biopotentials for diagnostic purposes, three different principles have emerged based on the type of sensor used, namely, sensors or sensor arrays adapted for implantation into the brain (invasive sensors), implantation into the skull and against the gray matter of the brain (partially invasive), or non-invasive placement meaning that the electrodes are simply placed on the skin. Invasive sensors have been used to alleviate the lack of functionality in individuals that suffer from some type of disability, for example, as described by Hochberg et al., “Neuronal ensemble control of prosthetic devices by a human with tetraplegia,” Nature 442: 164-171(13 Jul. 2006). Most invasive sensors are derivatives of the “Utah Array” developed by Richard A. Norman at the University of Utah, using approximately one hundred hair-thin electrodes to record extracellular potentials. In commercial applications, the Cyberkinetics “Braingate” is a device that uses invasively implanted electrodes to control wheelchairs and other devices. Likewise, partially invasive systems have already proven functional to play video games. In contrast, non-invasive electrodes have typically been limited to use for therapeutic purposes. As taught in U.S. Pat. Nos. 6,795,724 and 7,035,686, biofeedback using color-based neurofeedback has been employed based on the assignment of different colors on a computer screen to different states of neuronal activity.
Non-invasive electrodes generally need greater spatial separation for de-convoluting spatial properties of recorded signals as described in U.S. Pat. No. 6,014,582 or using near-field and far-field signals as described in U.S. Pat. No. 6,032,072. U.S. Pat. No. 6,950,698 discloses a five or seven electrode array and the positioning of the array on the forehead of a patient to optimally separate EOG, EEG and EMG signals. U.S. Pat. No. 7,206,625 to Kutz et al. discloses a compact measuring apparatus wherein the amplifier is directly adjacent to the sensors to reduce antenna effects and improve the signal to noise ratio. U.S. Pat. No. 6,728,564 discloses a system configurable to use a classical one-channel approach or else to alternately switch between predefined parts of the sensor array to simulate a two-channel system for EEG and EMG measurements. The Emotiv EPOC system employs a sensor array integrated into a helmet-like structure to convert the amplitudes of EEG signals into levitation of given objects in computer games and rotating the objects using rotational signals created by a gyroscope built into the headset.
A recurring issue associated with the use of biosignals is that it can be relatively difficult for a given user to control his or her brain activity. Alpha, beta and gamma brain waves are readily accessible for sensing with EEG sensors or related devices and can be separated into subgroups based on frequency properties. However, for most individuals it is very difficult to arbitrarily influence activity of selected subgroups of brain waves, especially in a time-controlled fashion. Timing of signals however is critical for most control functions, regardless of whether they are used for navigation systems or within another computer-related application. A case in point is the use of biosignals in gaming applications to trigger, for example, shooting or jumping in first person shooter (FPS) games.
In contrast to true brain waves, muscle signals can be readily and arbitrarily triggered, regardless of whether they relate to facial movements or, for example, eye movements. On the other hand, electrical muscle signals are difficult to separate into different channels, and tend to propagate across the body making it difficult to distinguish their precise origin. Even if accomplished, the user is posed with a somewhat difficult task of acquiring the necessary skills to master the exercise of different muscles without crossing over between groups.